Systems and methods for maximizing expected utility of signal injection test patterns in utility grids

ABSTRACT

Methods and systems for implementing experimental trials on utility grids. Variations in grid parameters are selected to introduce into utility grids to improve the value of learning from each experimental trial and promoting improved utility grid performance by computing expected values for both learning and grid performance. Those trials are used to manage the opportunity costs and constraints that affect the introduction of variations into utility grid parameters and the generation of valid data that can be attributed to particular variations in utility grid parameters.

BACKGROUND

The performance of utilities grids—their reliability, safety, andefficiency—can be drastically improved through sensing key parametersand using those results to direct the operations and maintenance of thegrid, by identifying faults, directing appropriate responses, andenabling active management such as incorporating renewable sources intoelectrical grids while maintaining power quality.

Sensor networks are often used to monitor utilities grids. These sensornetworks may include smart meters located at the ends of the grid,sensors at grid nodes, and sensors on or around the utilities lines,these sensors measuring grid parameters such as flow rates in watergrids, power quality in electrical grids, or pressures in utilitiesgrids. These sensors are transducers, usually outputting analog signalsrepresentative of the measured properties. These outputs need to becharacterized to map to specific values of those properties, and/orclassified so that they may represent particular states of the world,such as a potential leak that requires investigation, or identificationof a difference in phases when incorporating a renewable resource intoan electrical grid. Characterization of sensors is usually done throughbench testing, while the sensors may have various interferences in theenvironment surrounding them; in-situ characterization of sensors on autility grid monitoring network would be preferred, but is difficult forthe large numbers of sensors used to monitor a utilities grid.

The trend in analyzing sensor data and directing responses is “bigdata,” which uses large amounts of grid historical data to build modelsused for classification and direction of responses. These big datamodels, however, are limited to correlations, as they mine historicaldata to build the models, limiting their effectiveness for activelydirecting treatments or making fine adjustments. Further, these big datamodels typically require large volumes of data that prevent highlygranular understandings of grid conditions at particular grid nodes orlocations or that can only achieve such granularity after longoperations; some have applied machine learning techniques and improvedmodels to increase speed and granularity, but even these approachescontinue to rely on correlations from passively collected historicaldata.

Signal injections have been used to highlight grid faults, such asdiscovering nodes where power is being illegally drawn from an AC powergrid, or to test grid-wide response to large changes in high levels ofthe grid, such as at the HVDC distribution level. These signalinjections have been large, individual, and human mediated, and used toevaluate the system, not the sensors monitoring the system.

Utilities grid management would benefit greatly from real-timecause-and-effect understanding of sensor responses, remedying the issueswith big data smart grid approaches and allowing for real-time,granular, and fine-tuned grid monitoring and management.

SUMMARY

The present invention is directed towards methods for increasing thevalue of signal injections into a utility grid by receiving signalinjection characteristics for a plurality of potential signalinjections, receiving current sensor belief states, computing thelearning value of each of the plurality of signal injections, selectingsome of the plurality of potential signal injections based on thelearning values, and implementing those selected signal injections.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow diagram of the steps of a method of the invention.

FIG. 2 is a system diagram depicting an example of a system embodimentof the invention.

FIG. 3 is a data flow diagram of the flows of information among variouscomponents of a system of the invention.

FIG. 4 is a flow diagram of an example method for computing the learningvalues of different signal injections.

FIG. 5 is a flow diagram of an example method for determining theutility of signal injections and coordinating the signal injectionsaccording to their utility.

FIG. 6 is diagram depicting the architecture of system embodiments andtheir interactions with a utility grid.

DETAILED DESCRIPTION

Signal injections into utilities grids provide a valuable means ofcharacterizing sensors situated on or near a utility grid, anddiscovering utility grid response characteristics. However, the numberof potential signal injections may be limited by the need to ensure thatsignal injections that are concurrent do not interfere with one another;systems coordinating the injection of signals into a utility gridbenefit from a means of automatically identifying and implementing themost informative and/or lowest-opportunity cost signal injectionpatterns that can be made to improve efficiency in using limited timeand space to test and understand grid and sensor responses.

Signal injections to be made into utility grids are changes to gridparameters particular to those grids, such as voltage levels or waveforms in electrical grids, pressures and/or flow rates in gas grids,flow rates in water grids. The signal injections may be electricalsignal injections in such as increases or decreases in current, voltage,or power factor caused by actuating controls. The signal injection maybe implemented through automatic or human-mediated means. In gas grids,the signals may be injected through, for example, changing the routingof gas through pipes to increase or decrease the pressure at certainpoints. The responses to these signals may be the increase or decreasein the number and/or severity of leaks detected by a sensor networksurrounding the grid pipes, or changes in downstream pressures connectedto the areas being driven to high or low pressure. These signalinjections may be accomplished in human-mediated cases through themanual adjustment of various valves and switches at the direction of aschedule distributed to maintenance personnel who perform theseadjustment; these schedules may take various forms, such as maintenancequeues, additional tasks, and may be distributed through a variety ofelectronic means such as email, text message, calendar reminders on acomputer, tablet, smart phone or other portable computing device. Inthese human-mediated cases, the times of these adjustments may beaudited by having the maintenance personnel check in using a networkeddevice to record the time the changes are actually implemented, for usein the processing of subsequent data generated as a result of thesesignal injections. In fully machine-to-machine implemented embodimentsof signal injection on gas grids, the switches and valves are operatedby actuators coupled to the system through a wired or wirelesscommunications network, and responding to signals sent by the system oracting in accordance with instructions or schedules distributed to thecontrollers for those actuators by the system. Machine-to-machineimplementations allow for more closely coordinated tests as there willbe less variance in the time of implementation, and the improved timingallows more sophisticated trials to be conducted. In theseimplementations, monitoring of the sensor conditions and actuator statesmay be constantly correlated to create a real-time understanding ofrelationships among spatially and temporally distributed influences,enabling changes in relationships as well as local sensor states to bedetected and characterized, for example through factorial isolation ofdetected changes.

In electrical grids, human-mediated methods involve manual switching ofpower flow, activating or deactivating power sources connected to thegrid, adjusting the position of load tap changers, switching capacitorbanks on and off, activating or deactivating heavy industrial equipmentsuch as arc furnaces or other major manually-controlled major powerloads on the grid. In these examples, the changes are made by themaintenance personnel at the direction of a schedule distributed tothem; these schedules may take various forms, such as maintenancequeues, additional tasks, and may be distributed through a variety ofelectronic means such as email, text message, calendar reminders on acomputer, tablet, smart phone or other portable computing device. Inthese human-mediated cases, the times of these adjustments may beaudited by having the maintenance personnel check in using a networkeddevice to record the time the changes are actually implemented, for usein the processing of subsequent data generated as a result of thesesignal injections. These human-mediated methods may alter measurablefactors such as power quality, line temperature, line sag, availablepower levels, and other factors, which may be captured by sensornetworks observing those measurable grid factors.

In electrical grids, machine-to-machine methods offer a greater measureof control, and can inject signals through a variety of automated means.This includes automation of the types of switching and maintenancebehaviors that may be used in human-mediated examples such as changingthe position of load tap changers or switching capacitor banks, andadditionally M2M methods of signal injection may capitalize on greaterprecision and breadth of control to include actions such as coordinatinguse of devices such as appliances at end locations to create coordinateddemand and loading at consumer locations, or to implement complexcoordination of combinations of multiple types of grid-influencingactions to generate more complex conditions, or introducing changes intothe automatic power factor correction units. These combinatoricpossibilities are very difficult to address through big-data approaches,since even large volumes of data may only have limited sample sizesreflecting particular combinations, and the sheer number of combinatoricpossibilities makes big data solutions to these problems nearlyintractable. These may be initiated through automatic control of theassociated grid components and networked devices, including powergeneration, switches, voltage regulation equipment, smart meters andsmart appliances receiving power from the grid, and other gridcomponents susceptible to remote control by the system. These may takeadvantage of millisecond-level control capabilities to manipulate powerquality variables such as the integration of new sources or immediateresponses to new loads or the specific operation of automatic powerfactor correction units, as well as further increase the ability to testcombinatorics of grid actions or conditions involving those highlytime-sensitive variables.

Signal injections may be selected for their potential to falsify currentmodels of sensor response, to characterize the sensor responses (forexample, that a particular level of output from the sensor is indicativeof a particular level of the sensed variable) or to classify the sensorresponses as indicative of a particular event either categorically (forexample, in a water grid, that particular sensor output signals from twosensors are indicative of a severe leak being present) orprobabilistically (for example, in a gas grid, that a particularelectrical output from a methane sensor is 60% likely to indicate aCategory 3 leak, 30% likely to indicate a Category 2 leak, and 10%likely to not be indicative of a leak). Grid responses to perturbationsof known type and magnitude allow for the testing and potentialfalsification of these models, allowing systems to converge oncharacterizations or classifications for raw sensor outputs that arebased on their in-situ performance and readings, streamlining theprocess of sensor characterization for detecting events and states ofutility grids.

The injected signals may be simple, directing one grid action such asopening a valve in a water or gas grid, or bringing one particularrenewable source online or altering the output voltage from onesubstation in electrical grid examples to induce the desired, controlledchange to grid conditions, or they may be complex, composed of multiplegrid actions coordinated such that their individual spatial and temporalreaches overlap to produce a multi-factor treatment at areas within theoverlapping reaches. One example of a complex grid action may be to varyboth load tap changer positions and capacitor bank switchingsimultaneously to provide more fine-grained control over reactive powerin an electrical grid. This multi-factor treatment may include variancesof multiple different grid parameters, for example to explorecombinatoric effects of those parameters, or may be used to producemultiple instances of similar variations of a particular grid parameter,for example to use additive effects to increase the magnitude of aparticular variance of a grid parameter at one or more specificlocations on the grid while protecting more sensitive neighboring partsof the grid by keeping them within narrower or different operationalranges by exposing those parts to only a component of the overall signalinjection.

For complex signals, the temporal and spatial reaches are predictedbased on treating the complex signal's effects on the system as a whole,composed set. For those complex signals, while individual grid actionswill have overlapping spatial and temporal reaches, the defined set ofgrid actions that make up the complex signal is instead treated as onesignal injection, with the overall spatial and temporal reach of thecombination of the defined set of grid actions used to determine theareas of space and periods of time where no other signals may beinjected into the grid, to maintain the orthogonality of the complexsignal injection from other grid signal injections.

Complex signals may be input into the system having already been definedas the set of grid actions to be done together and the times andlocations of those grid actions, after being derived by other systems orselected by grid personnel, or may be derived by systems selectingmultiple grid actions from the set of grid actions as directed by, forexample, a Partially Observable Markov Decision Process (POMDP) modelexploring combinatorics or operating within constraints on operationalconditions that vary from location to location across the grid.

Signal injections exploring grid responses may be composed by searchingfor waveforms that have a spatial-temporal regularity with anycontrolled grid activity, which are co-occurring in immediate or regulardelayed fashion, for example through Principal Component or Fourieranalysis. These statistical regularities in waveforms or componentwaveforms (for example, the frequency, voltage, and/or current) linkgrid actions with changes in grid conditions to provide the set ofavailable options for manipulating grid conditions based on activecontrol of grid actions and data on the observed times and locations ofthese waveform components relative to the grid actions may be used todetermine spatial and temporal reaches for particular signal injections.

FIG. 1 is a flowchart outlining a method embodiment of the invention.Signal injection data is received in step 100 and current sensor beliefstates are received in step 102. The sensor belief states are used alongwith the signal injection data to compute learning values for signalinjections in step 104. Costs and benefits for signal injections arereceived in step 106. Signal injections are selected and coordinatedbased on computed values in step 108, and the signal injections areimplemented on the utility grid in step 110. Sensor data may becollected from a sensor network on the utility grid in step 112, andthat collected sensor data used to update models of sensor response,such as classifiers, probability estimates and/or characterizationmodels in step 114.

Signal injection data received in step 100. The signal injection data isthe time, location, and attributes of the signal to be injected into theutility grid, with the attributes of the signal injection including, forexample, the changes made to the grid to implement the change, or themagnitude of the signal being added and the type of the signal. Thesignal injection itself is a change in grid controls affecting gridparameters. In an electrical grid, an electrical signal injection may bean increase or decrease in voltage, current or power factor resultingfrom the change in state of a control. For example, a signal injectionin an embodiment of the invention directed to water distribution gridsmay have a nature described by the closing of two valves at one node onthe water distribution grid and the opening of another at an adjacentnode. The attributes of the signal injection indicate what gridparameters are likely to be altered by the signal injection, with thesignal injection being a particular selection of grid controls from theordinary operational ranges of those grid controls. This may in turn beused to determine which sensors would have their outputs affected by thesignal injection. For another example, a signal injection in anembodiment of the invention directed to electrical grids may have itsattributes described as the addition of reactive power at a substation,implemented by switching on a number of capacitor banks. The location ofthe signal injection may be given in terms of a grid location, such asthe particular valves, lines, transformers, substations, or sources thatwill be used to implement the signal injection, or geographiccoordinates where the signal injection will be implemented.

Current belief states are received in step 102. Steps 100 and 102 may beperformed simultaneously or in either order, with step 100 preceding orfollowing step 102. The belief states are a set of different models ofsensor response, each model corresponding to a relationship between thesensor output and the events or world states acting on the sensor toproduce that output. These models may each be, for example, classifiersmapping the sensor outputs to specific world events or states,probability estimates mapping the sensor outputs to a plurality ofpossible world states, or characterization models mapping sensor outputsto particular levels of a sensed variable. These belief states may haveattached uncertainty values reflecting the likelihood that they areaccurate given the current set of trials and knowledge that may tend toconfirm or falsify these different models, and the information that canfurther confirm or falsify the models may be included in this data orderived from the basic characteristics of the particular model.

The learning value that a signal injection can provide, for example byreducing the uncertainty around the current set of belief states iscomputed in step 104. The learning value is a measure of the value thatknowledge generated as a result of the signal injection may provide tosubsequent decision-making by a system, such as value that could beprovided by reducing uncertainty in a sensor measurement, or determiningthat a particular action is more likely to be optimal. The learningvalue may be computed through, for example, predicting the raw number ofbelief states that may be falsified according to the predictions of aPartially Observable Markov Decision Process (POMDP) or otherstatistical model, predicted impacts of the signal injection on theuncertainty levels in the belief states in such models, or experimentalpower analyses computing the reduction in uncertainty and narrowing ofconfidence intervals based on increasing to the current sample size. Fora particular example, a Bayesian Causal Network may be used to identifydependencies in the data to discover potentially valuable trials thatmay efficiently refine a grid control system's knowledge of sensorresponse. Systematic multivariate experimentation is done to analyze thedirectionality and variables involved in the underlying causal paths forthose waveform components, by going back to the normative operationalconstraints and using constrained randomization, and experimentaldesigns (such as Latin Square) to systematically explore which gridcontrol elements and combinations thereof are the underlying cause ofthe waveforms. These experimental designs may be iterated to refine theanalysis, for example eliminating three-fourths of the controls on abasic first pass, through elimination of those controls that are randomwith respect to the waveform components of interest, and then usingfactorial combinations of the remaining controls in a second trial toproperly identify the control or combination of controls causally linkedto those waveform components of interest.

An example of one method for computing the learning value of a signalinjection is presented in FIG. 4. Current signal injection response datais received 400, a predicted change in confidence intervals through anadditional signal injection is computed 402, changes in optimal behaviorare computed for the predicted change in confidence intervals in step404, and the utility of the predicted changes are computed in step 406.

The current signal injection data is received 400. The current signalinjection may be, for example, a table of inferential statisticsdescribing the relationship between a particular signal injection andthe response of sensors during times and locations associated with thesignal injection. This may take the form of a mean response andconfidence intervals for the response.

For the signal injection data, a predicted change in confidenceintervals for a signal injection is computed in step 402. This may becomputed through an experimental power analysis to determine thereduction in confidence intervals by increasing the sample size comparedto the current signal injection data.

The predicted confidence intervals are used to compute a predictedchange in behavior in step 404. The current overlap in confidenceintervals may be used to determine the relative frequency of actions orthe relative weight of competing models of sensor response. Changes tothe size of the confidence intervals based on a particular signalinjection, as computed in step 402 through power analysis and theincrease in sample size, will alter the overlap in the confidenceintervals. A prediction of the change in the relative frequencies can becomputed using the optimization module that selects among or weights thedifferent actions or models of sensor response.

The utility of the predicted changes is computed in step 406, based onthe predicted change in relative frequencies and the predicted outcomesof the actions using the predicted confidence intervals. The utilitycomputed in step 406 is output for use as the learning value of thesignal injection, representing the value that can be extracted from theknowledge gained by making a particular signal injection as part of acoordinated set of signal injections used to perturb a utility gridwhere automated experimentation is used to improve the efficiency whentesting grid response and associated sensor response.

The learning value may also be modified by the potential value ofincreasing the particular type of knowledge that the trial will examineor the utility of further refining the models being tested. The metricsaffected and models refined through signal injections may differ in typeand therefore offer different values to grid operators, such as onesignal injection improving fault detection while another would refineknowledge for demand reduction on an electrical distribution grid, orthere may be non-linearities in the value that additional learning orrefinement of grid and sensor response models provides to gridoperators. For example, in gas grids, the number of small leaks vastlyoutpaces the ability of maintenance resources to address the smallleaks, so improvement in localization of Category 1 leaks may provideless value to gas grid operators than improvements in the identificationof leaks that are likely to worsen over time. This may be represented byutility functions that incorporate the value of the type of learningalong with the magnitude of the learning, or predictions based on modelsused to plan grid responses to mitigate harm or increase efficiency, andproject the additional cost savings that they can determine withimproved data having reduced uncertainty values. For example, a capitalplanning module for grid improvements may derive one set of values foran equipment replacement problem on a utility grid given the current setof data, but alternative data sets based on reductions in uncertaintythat are possible through additional trials may be input into themodule, and the differences between the current and reduced-uncertaintycases used to estimate a value for the potential uncertainty reductionthat may be realized through implementing trials through particularsignal injections into the utility grid.

Cost Information is received in step 106; this may be computed from thesignal injection characteristics and model data, based on the details ofimplementing the signal injections included in the signal injection datareceived in step 100. The costs of the trials includes the actual costto generate the signal and observe the response, for example reductionsin flow rates leading to less chargeable distribution of water or gas tocustomers of those grids, or the cost of deploying a maintenance crew toan area to implement a human-mediated signal injection, and may alsoprice in risks associated with signal injections, such as increased riskof the sensor network missing particular events occurring within thespatial and temporal uncertainties for the trial due to the sensoroutputs being driven by the signal injection, or potential disruptionsto scheduled maintenance due to temporal uncertainty regarding signalinjection duration and the need to avoid interfering with trials, or useof resources to introduce human-mediated signal injections. This may bedone, for example, by discounting a projected cost of the risk event bya projected increase in likelihood of that event by the implementationof the signal injection, creating an expected value for the added riskintroduced by the signal injection.

The computation of the costs for signal injections may vary based on thelocations and periods of time captured within the spatial and temporalreach of the signal injection and the time and location where the signalinjection is implemented. These opportunity costs may be predictedthrough use of normative operational condition data that includes localgranularity, such as different tolerance ranges for different nodes orgeographical segments along the grid or during particular periods ofhigh or low stress on the grid, through component modeling reflectingthe ages, maintenance conditions and types of components making up thegrid infrastructure in different locations across the grid, and/or theuse of grid state or use data to determine opportunity costs over time.For example, power quality baseline standards used to determine theextent of deviation from those standards may vary depending on thecharacterizations of the electrical grid users drawing power at aparticular place and time, to account for the different sensitivities ofdifferent devices to aspects of power quality, for example, thesusceptibility to electrical noise of computers as opposed to lighting,whose use may vary with the types of users and the times at which usersdraw power from the grid. For example, an area known to contain datacenters may have a higher coefficient that is used for calculating thecost associated with increases in electrical noise, specific to makinglocal determinations of the cost of signal injections which arepredicted to impact the amount of electronic noise experienced by thatportion of the grid. Cost data may also include benefits of the signalinjection based on the expected effects of the signal injection on thegrid parameters and the desirability of those changes in gridparameters. The combined cost and benefit data provides an expectedeffect value which may be combined with the learning value to determinea value for the signal injection. The expected effect value is aprediction of the value of a signal injection based on the impact ofthat signal injection on grid parameters. The expected effect value maybe positive or negative, representing improvement or degradation in gridperformance metrics. For an example of a negative expected effect value,a signal injection that is predicted to reduce power factor delivered byan electrical grid will have an expected effect value based on theexpected reduction in power factor, modified by a weighting factor thatrepresents the cost created by the reduction in power factor.

The learning values and cost data are used to compute utilities andcoordinate signal injections based on those utilities in step 108. Theutility for a particular signal injection is determined through autility function that incorporates the value of implementing aparticular signal injection and the improvement in knowledge likely toresult from it from step 104 against the potential costs and risks ofimplementing the signal injection detailed in the cost data from step106, modified by other factors and converted to common metrics which maybe arbitrary or based in values such as currency. Signal injections arecoordinated such that the signal injections remain orthogonal to oneanother through ensuring that they do not have spatial and temporaloverlap in the areas they are expected to have observable influence;this may be done through using historical data associating gridconditions with grid actions, such as by identifying waveform componentsin the electrical waveform on a power grid through Fourier or PrincipalComponent Analysis that are associated with the grid actions that makeup the signal injection, or using other models of grid characteristicssuch as the current belief states or component models and physicalcharacteristics to predict the spatial and temporal reach of the signalinjection. The coordination of these signal injections may be donethrough graphical modeling techniques such as Bayesian networks orMarkov random fields or subspecies thereof. The coordinated signalinjections are selected to maximize the computed utility over time; thismay be done as the signals are coordinated through the graphical model,or may be done by coordinating multiple possible sets of signalinjections using the graphical model, finding the sum utility over timefor each and selecting the set of signal injections from those multiplepossible sets based on that aggregate utility. Calculating utility forfull sets of signal injections across the grid allows those embodimentsof the invention to capture the opportunity costs inherent in the needto maintain orthogonality among the signal injections, since each signalinjection necessarily limits the other potential signal injectionsthrough those temporal and spatial reaches which may not overlap.Selecting signal injections and implementing them into the grid in thismanner increases the efficiency at improving the understanding of sensoroutputs along the utility grid by automatically managing numeroustradeoffs and opportunity costs existing where signal injection spatialand temporal reaches may not overlap.

One example of a process used to select a set of signal injections toimplement on the utility grid is presented in FIG. 5. A plurality ofsets of signal injections are generated 500, a set of signal injectionsis selected from that plurality 502, the utility for that set of signalinjections is computed 504 and compared to the utility of the set ofsignal injections scheduled for implementation 506. If the selected sethas higher utility than the scheduled set, the selected set replaces thescheduled set 508. If the selected set does not have higher utility, itis rejected 510. Either way, the process is iterated for all of theplurality of sets of signal injections until each has been evaluated.

A plurality of sets of signal injections are generated in step 500. Thismay be done through methods such as graphical models, Bayesian Causalnetworks, or other methods generating a set of permissible signalinjections which do not have overlap in their spatial and temporalreaches. From this plurality, an individual set is selected 502. Thisselection may be randomized or done in some sort of sequential order.Using the learning values and costs for each signal injection that isincluded in the set, utilities are computed for each signal injectionand summed together to produce the utility for that set of signalinjections to compute the utility for the selected set of signalinjections 504.

For the first selected set of signal injections, the baseline utility iszero, so that set is accepted as the signal injection set to bescheduled for implementation in accordance with step 508. For all otherselected signal injections in iterations of this example, the utility ofthat selected signal injection is compared to the utility of the signalinjection set scheduled for implementation 506. If the selected signalinjection set has a higher utility than the scheduled signal injectionset, the selected signal injection set is accepted 508, by making theselected signal injection set the new scheduled signal injection set,discarding the prior scheduled signal injection set. If the selectedsignal injection set has a lower total utility than the scheduled signalinjection set, the selected signal injection set is rejected 510 bydiscarding the selected signal injection set. After either acceptance orrejection of the selected signal injection set, the process is iteratedby selecting a new signal injection set from the plurality 502, untilall of the signal injection sets in the plurality have been tested. Thefinal scheduled signal injection set at the end of this process isimplemented into the utility grid to perturb the grid to efficientlyproduce knowledge that is used to drive subsequent operations or improveinterpretation of sensor responses.

Returning to FIG. 1, the selected signals or combination of signals isthen injected into the appropriate locations on the sensor network instep 110. The signals are injected into the sensor network according tothe coordinated set of signal injections and upholding their temporaland spatial uncertainty constraints, by taking the directed grid actionsat the proper times and locations. The signal injections may beimplemented by human actors, such as grid maintenance personnel, bydirecting them to perform the grid actions such as operating switches inelectrical grids, or opening and closing valves on water and gasdistribution grids, through distributing appropriate instructions tothose grid personnel through means such as email systems, automatedmessaging, queuing systems, or other means of instructing the humanactors on what actions to take to influence the grid and when and whereto implement them. The signal injections may also be partially or whollyimplemented through machine-to-machine actions, such as havingprocessors direct the actions of actuators controlling switches andvalves, or controllers automatically directing the activation ofrenewable sources or otherwise implementing the directed grid actions,based on signals and/or data distributed to those processors andactuators, switches, sources and other grid components detailing thegrid actions to take and the time and location for those grid actions tobe taken. The injection of these signals perturbs the utility grid,enabling more efficient generation of relevant knowledge about grid andsensor response by controlling opportunity costs of different signalinjections that produce different types and amounts of knowledge whileoften being mutually exclusive due to confounding and the need toattribute particular sensor responses and grid events with particularsignal injections.

Monitoring sensor responses to the signal injection may be done at step112. The sensors are distributed across the utility grid, and may beintegrated with the grid, in examples like sensored cables andterminations on electrical grids, placed on the grid such as the sensorsincluded in smart meters on electrical grids, or may be placed inproximity to the grid such as methane sensors in gas distribution grids.The sensors typically are transducers that produce an electricalwaveform as output when exposed to the sensed variable, although thiselectrical response may be partially or wholly non-linear. The sensoroutputs may have metadata associated with the outputs to provideindications of the time and location where the signal is collected, suchas a time-stamp and an identification number for the sensor which can becross-referenced with a database of the sensor numbers and theirlocations, allowing the sensor output data to be parsed by time andlocation to associate them with particular signal injections based onthe reach of those signal injections and the time and location at whichthe signal injections were implemented.

The belief states may be updated based on the sensor responses and thesignal injection properties in step 114. The signal injectioncharacteristics and the sensor data associated with the signalinjections are used to test and confirm or falsify the related beliefstates. In one example, this may be done through comparison of theassociated signal injection output data and predictions made regardingeach belief state model makes regarding the response the model wouldexpect to that signal injection based on its characteristics. In thisexample, the model predictions of sensor response, derived based on thesignal injection characteristics and the models for each belief statebeing tested, are compared with the actual associated response of thesensors to the signal injection; based on the accuracy of thepredictions, models may be falsified depending on the extent to whichthey deviate from the real observed values. Associated sensor data mayalso be used to update the means and reduce the size of the confidenceintervals associated with data concerning grid responses to particulargrid actions taken in the associated signal injections, or added todatabases of historical knowledge used as the basis for sensorcharacterization models in some example embodiments of the invention,improving the precision and accuracy of sensors whose raw outputs areclassified or characterized through these improved models of sensorresponse.

FIG. 2 is a diagram of an example embodiment of the invention as acoordinated utility grid system. Memories may be known computer storagemeans such as flash memory, hard disk drives using magnetic media, orother methods for data storage that can store the data and be accessedfrequently and regularly. Processors may be configured to make thecalculations through software instructions. Connections among thecomponents may be hard-wired, use of common processors for multiplesteps, or networked through wired or wireless means such as the various802.11 protocols, ZigBee or Bluetooth standards, Ethernet, or other suchmeans for transmitting data among the separate sensors, processors,memories and modules. The sensors, memories, processors, and modules maybe distributed across locations, including at the sensors or on gridlocations themselves, or co-located in intermediate or centrallocations.

Signal injection memory 200 stores the characteristics of signalinjections that may be made into the utility grid. This memory isconfigured to store the characteristics of potential signal injections,including the time, location, magnitude and parameters being affected bythe signal injection. This memory may also store implementation data forthe signal injection, such as the set of instructions to be presented togrid personnel for human-mediated embodiments, or the actuators andcommands to be distributed to them in machine-to-machine embodiments ofthe invention.

Belief State Memory 202 stores the current set of belief states. It maybe a database containing the models that are used to classify orcharacterize sensor outputs, such as classifiers, probability estimates,and models mapping the output signals of the sensors to the transducedvariables at the location of the sensor. These belief states may alsoinclude other factors or metadata representative of the level ofcertainty regarding the accuracy of the model, or the types of signalsthat may confirm or falsify the accuracy with which the model properlyrepresents what is being sensed based on the sensor's output signal.

Cost memory 204 stores information on the potential costs of the signalinjection. This may be a database of potential grid actions combinedwith a set of costs associated with that particular grid action,representing the cost and risks associated with including that gridaction as part of a signal injection that may be implemented. Such costsinclude potential losses such as reductions in chargeable provision ofthe utility due to reductions in flow to certain areas, the cost ofdispatching grid personnel to implement the changes, risks of deviatingfrom normative operational parameters, or loss of some sensing abilitiesbecause of the signal injection overwhelming or masking other changes inthe sensed variables at sensors. This data may be organized such thatparticular grid actions are valued differently at various times andlocations due to differences in the costs to be expected for thosedifferent implementations, such as discounting the loss of potentiallychargeable utility distributions at times where demand would be met bythe diminished flow of the utility, or by region such as havingdifferent renewable sources of the same type having different costs topower quality disruptions they introduce because of different localmarkets they serve that differ in sensitivity to that power quality.

Learning Value processor 206 computes the expected value that can beassociated with falsifying belief states or improving confidenceintervals used in models representing grid conditions detected by gridsensors for a particular signal injection. The learning value processor206 may compute the number of belief states confirmed or falsified by aparticular trial based on the current values of the belief states andthe characteristics of the signal injection, may use power analysis topredict the reduction in uncertainties to result from increases in thesample size, or discovery of dependencies in the data. The learningvalue processor 206 may be configured to apply POMDP or Bayesian CausalNetworks, for example, to determine these values. The learning valueprocessor 206 may optionally be configured to account for differences inthe relative value of types of knowledge about grid conditions, ornon-linearities in the value of such knowledge, for example by applyinga utility function or applying modification factors to differentquantities representing the potential reductions in uncertainty orbelief states to be falsified, based on the nature of those learningsand the potential improvements in grid operation that can be expectedfrom such learnings.

Selection Processor 208 coordinates signal injections to maintainorthogonality using the spatial and temporal reaches of the signalinjections and generates a set of coordinated signal injections based onthe expected utility of the set of signal injections that directs theimplementation of those signal injections into the utility grid. TheSelection Processor 208 may be configured to coordinate of these signalinjections by applying graphical modeling techniques such as Bayesiannetworks or Markov random fields or subspecies thereof to ensure thatthe spatial and temporal reaches of signal injections arenon-overlapping. The Selection Processor 208 may be configured to applya utility function to the learning value and the associated costs of asignal injection to determine the signal injection's utility, which isused in creating a final coordinated set of signal injections to theutility grid.

Injection Implementation Modules 210 may be tools for distributing andensuring compliance with instructions governing the signal injectionsand their coordination across the utility grid in human mediatedembodiments, and/or may be processors, controllers, and actuators usedto automatically implement the signal injections in machine-to-machineembodiments of the invention. Examples include, for machine-to-machineexamples, actuators controlling valves in water and gas grids, controlcircuits and actuators for load tap changers situated at electricalsubstations, switches controlling connections between distributed powersources such as solar or wind generators and the remainder of the grid,or switches for capacitor banks in electrical distribution grids. Forhuman-mediated embodiments, examples include automatic generation anddistribution of emails or text messages, computing devices carried bymaintenance personnel and the servers they sync to for receiving queuinginstructions and reporting completion of tasks such as taking actionsthat implement signal injections and status of the grid and/orcompletion of assigned maintenance tasks.

Grid sensor network 212 may be a plurality of sensors distributed acrossthe utility grid to measure grid parameters, such as flow rates,current, voltage, line temperature, line sag, and whose output mayreflect the changes in grid conditions resulting from signal injections.These sensors may be, for example, methane detectors, sensored cableterminations, water flow meters, electrical “smart meters”, or othersuch grid sensors. These sensors monitor changes in grid conditionsstemming from the implemented signal injections, and that data may beparsed according to the spatial and temporal reaches of the signalinjections based on the time and location at which the sensor capturesthe data.

FIG. 3 is a data flow diagram showing an example embodiment of theinvention as a coordinated utility grid system and outlining thegeneration, flow and transformation of data by various system elementsand actions taken by system elements.

Signal injection properties 300 are data describing the signalinjections that may be made on the grid, including factors such as thelocation and magnitude of such signal injections, the grid actions thatare performed to implement each signal injection. This information isstored in the signal injection memory 302, and transferred to thelearning value processor 304 so that the signal injection properties maybe used to derive the learning value 306 of that signal injection, andthe selection processor 308 to be coordinated and chosen for utility toproduce the signal injection selection 310.

Belief States 312 are a set of models that potentially describe therelationship between sensor outputs and grid conditions, such asclassifiers, probability estimates or characterization models. Thesemodels also may include metadata concerning the certainty of the models,the historical performance of the models, and/or the information that islikely to confirms or falsify those models. It is stored in belief statememory 314 and is transferred to the learning value processor 304 sothat the impact of signal injections on the number and/or certainty ofbelief states may be computed. The belief states may also be updatedbased on parsed sensor data associated with particular signalinjections, based on the extent to which the parsed sensor data matchespredictions of response to the signal injection made by each beliefstate model.

Learning values 306 represent the value of learning associated with aparticular signal injection, and are computed by the learning valueprocessor 304 based on the signal injection properties 300, the beliefstates 312 and optionally may include scaling factors or be based onutility calculations that account for the value of particular fields oflearning, based off of relative values among and non-linearities withinthe value of increasing knowledge of grid and sensor responses. Thelearning values 306 are transferred from the learning value processor tothe selection processor 308 where they are used as a basis for computingsignal injection utilities when generating the coordinated signalinjection selection 310.

Cost Values 318 are data representative of the costs and risksassociated with implementing signal injections, such as the ordinarycosts of implementation such as maintenance personnel tasks, or reducedopportunities to provide customers with the utility, risks associatedwith temporary loss of sensor sensitivity or of departing from normativeoperational constraints. They are received, such as from user input orfrom databases containing the cost information, or derived from signalinjection properties 300 and stored in cost memory 320, and transferredfrom cost memory 320 to selection processor 308 to be used in computingsignal injection utilities that are used to generate the coordinatedsignal injection selection 310.

Signal Injection Selection 310 is derived at the selection processor 308and is the coordinated set of signal injections that is then distributedto the signal injection module or modules 316, directing the gridactions such as, for example, switching of capacitor banks, activationof distributed generation resources, or adjusting the pressure of gas ina line, that will implement the coordinated signal injections andperturbing a utility grid to generate data that may be used to reducethe uncertainty in grid models and/or improve the belief states 312 thatare used to characterize or classify sensor responses of sensors on thegrid.

Sensor data 322 is raw waveform outputs from transducers that measuregrid-relevant metrics such as line temperature, line sag, voltage,current, gas or water flow rates, or gas pressures that are collected bysensor network 324 that is situated in, on or near the utility grid. Thesensor data may be parsed by the time and location of its collection toassociate it with particular signal injections, and that associatedsensor data may be used to validate and confirm or falsify some beliefstates 312. The sensor data may also be used with the belief states 312to create a representation of grid conditions and guide active gridcontrol and management efforts such as fault identification, faultrestoration, management of power quality, grid capital planning,renewable source integration, or improving grid component longevitythrough grid parameter management.

A simple example of an overall architecture involving an exampleembodiment of the invention is presented in FIG. 6. The control decisionlayer 600 makes decisions about the states for some or all girdcontrols. Grid control decisions are made according to methods ensuringthat the manipulation of controls creates samples that do not influenceone another, and optionally selecting the control decisions to providehigh learning value or to improve particular grid parameters such asensuring certain voltage levels in electrical grids, or flow rates ingas or water grids. The control decisions from the control decisionlayer 600 are carried out by the controls 602, 604, and 606. Examples ofparticular controls include capacitor bank switches, load tap changers,switches and storage devices on electrical grids, or valves and sourceson water and gas grids. The controls may carry out the control decisionsby, for example, actuating switches, moving load tap changer positions,and narrowing or widening valves. The actions of the controls changegrid parameters, and those changes propagate through the grid 608. Forexample, opening a valve on a gas grid may cause pressures to increasedownstream over time, within a certain distance from the valve, or in anelectrical grid, power quality and reactive power levels may changebased on the switching on or off of a capacitor bank. Sensors 614, 616,and 618 placed along the grid measure grid parameters, and detect thepropagation of the signal injection through the grid 608. The signalinjections are limited in the extent to which they propagate through thegrid 608, defined as the spatial reach of that signal injection such asthe spatial reach 610 outlining the region affected by the signalinjected by control 602 and including the connection of sensor 614 tothe grid 608, and spatial reach 612 outlining the region affected by thesignal injected by control 606 and including the connection of sensor618 to grid 608. Data processing layer 620 associates the data fromsensors 614, 616, and 618 with signal injections whose spatial andtemporal reaches include the sensor data, for example associating datafrom sensor 614 with data from a signal injection implemented by control602 based on spatial reach 610, and associating data from sensor 618with a signal injection implemented by control 606 based on spatialreach 612. The associated sensor data from the data processing layer 620is then analyzed by the data analysis layer 622 to determineunderstandings about grid behavior and sensor response. Thisunderstanding of grid behavior generated by the data analysis layer 622may, for example, take the form of sensor response models which are usedto interpret the outputs from grid sensors 614, 616, and 618 duringordinary operations, for example to set thresholds or alerts forbrownout conditions when voltage drops in an electrical line, or settingan alert for methane levels crossing normal operational thresholds. Thedata analysis layer 622 may interface with the control decision layer600 to iteratively coordinate and implement signal injections into thegrid and provide information that improves the selection of signalinjections to implement, for example by predicting the effects of asignal injection on the grid or computing the extent to which learningmay be refined by a particular signal injection.

The invention claimed is:
 1. A method for injecting signals into autility grid, comprising: receiving a spatial reach and a temporal reachfor each signal injection of a plurality of signal injections; computinga learning value for each signal injection in the plurality of signalinjections; computing an expected effect value for each signal injectionin the plurality of signal injections; selecting, based on the learningvalues and expected effect values, a set of signal injections whereinthe spatial reach and temporal reach of each signal injection do notboth overlap the spatial reach and temporal reach of another signalinjection in the set; and injecting the selected set of signalinjections into a utility grid, wherein the signal injections arechanges in the state of grid controls, wherein the grid controls are atleast one of load tap changers, switches, and storage devices onelectrical grids.
 2. The method of claim 1, further comprising measuringthe utility grid response to the injected signals.
 3. The method ofclaim 2, further comprising associating data from the sensors withsignal injections based on the time and location of the sensor data andthe spatial reach and temporal reach of the signal injections.
 4. Themethod of claim 1, wherein the learning value is computed based on anumber of belief states that may be falsified by grid response to thesignal injection.
 5. The method of claim 1, wherein the learning valueis computed based on a predicted change in the width of confidenceintervals for grid response to the signal injection.
 6. The method ofclaim 1, wherein the expected effect value is computed based on adatabase of the effects of prior signal injections.
 7. A method forinjecting signals into a utility grid, comprising: receiving a spatialreach and a temporal reach for each signal injection of a plurality ofsignal injections; computing a learning value for each signal injectionin the plurality of signal injections; computing an expected effectvalue for each signal injection in the plurality of signal injections;selecting, based on the learning values and expected effect values, aset of signal injections wherein the spatial reach of each signalinjection overlaps the temporal reach of each same signal injection; andinjecting the selected set of signal injections into a utility grid withthe overlapping spatial and temporal reaches, wherein the signalinjections are changes in the state of grid controls.
 8. The method ofclaim 7, further comprising measuring the utility grid response to theinjected signals.
 9. The method of claim 8, further comprisingassociating data from the sensors with signal injections based on thetime and location of the sensor data and the spatial reach and temporalreach of the signal injections.
 10. The method of claim 7, wherein thelearning value is computed based on a number of belief states that maybe falsified by grid response to the signal injection.
 11. The method ofclaim 7, wherein the learning value is computed based on a predictedchange in the width of confidence intervals for grid response to thesignal injection.
 12. The method of claim 7, wherein the expected effectvalue is computed based on a database of the effects of prior signalinjections.
 13. A method for injecting signals into a utility grid,comprising: receiving a spatial reach and a temporal reach for eachsignal injection of a plurality of signal injections; computing alearning value for each signal injection in the plurality of signalinjections; computing an expected effect value for each signal injectionin the plurality of signal injections; selecting, based on the learningvalues and expected effect values, a set of signal injections whereinthe spatial reach and temporal reach of each signal injection do notboth overlap the spatial reach and temporal reach of another signalinjection in the set; and injecting the selected set of signalinjections into a utility grid, wherein the signal injections are atleast one of changing valves, pressures, and flow rates in a gas grid.14. The method of claim 13, wherein multiple signal injections areconcurrently implemented on the utility grid.
 15. The method of claim13, wherein the signal injections are coordinated by a PartiallyObservable Markov Decision Process.